Skip to content

Apply DefaultIfEmpty where a leftOuterJoin group is read - #20903

Draft
xperiandri wants to merge 12 commits into
dotnet:mainfrom
xperiandri:fix/query-left-outer-join
Draft

xperiandri wants to merge 12 commits into
dotnet:mainfrom
xperiandri:fix/query-left-outer-join

Conversation

@xperiandri

Copy link
Copy Markdown
Contributor

Draft, do not merge. SQLProvider 1.5.27 finds a left join by the DefaultIfEmpty inside the GroupJoin result, which is what F# emits today. With this PR every leftOuterJoin there silently becomes an INNER JOIN (16 queries and 8 of its own tests), and EF Core stops translating a group that is both iterated and read again. Kept for discussion of a design that works for both, for #6552.

Stacked on #20902

Two leftOuterJoins in a row now become two SQL LEFT JOINs: the group reaches the rest of the query as GroupJoin produced it, and DefaultIfEmpty is applied in the SelectMany that reads it, the shape C# produces and EF Core and linq2db recognize as a left join.

Details
  • leftOuterJoin used to apply DefaultIfEmpty inside the GroupJoin result, so the join's element carried an already-defaulted group. With a second leftOuterJoin after it EF Core needed a lateral join (APPLY), which SQLite lacks, and linq2db failed.
  • The elements are the same, but the group is no longer one object per joined row: each read makes its own DefaultIfEmpty sequence. A query that compares the group itself by reference over LINQ to Objects, such as select g; distinct or groupBy g, sees the difference.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

✅ Release notes checked


✅ Found changes and release notes in following paths:

Change path Release notes path Description
`src/FSharp.Core` docs/release-notes/.FSharp.Core/11.0.200.md

@xperiandri

Copy link
Copy Markdown
Contributor Author

Covered cases

EF Core SQLite translates 102 of 139 scenarios, 100 on #20902. Now translated: two left joins spelled leftOuterJoin, and a join after a left join.

linq2db translates two leftOuterJoins too; it fails on the second today. EF Core for SQL Server and PostgreSQL produces two LEFT JOINs where it used a lateral join.

Why this is a draft

  • SQLProvider 1.5.27 on SQLite: every leftOuterJoin becomes an INNER JOIN without an error, so rows with no match go missing (16 queries and 8 of its own tests). It sets isLeftOuter from a DefaultIfEmpty inside the GroupJoin result selector, and treats a DefaultIfEmpty in the SelectMany collection selector as a no-op, which is the C# shape this PR produces.
  • EF Core: leftOuterJoin o in db.Orders on (…) into orders; for o in orders do select (c.Name, orders.Count()), a group that is both iterated and read again, translates on main and no longer does.

xperiandri and others added 12 commits November 8, 2026 02:46
Every query test so far runs over AsQueryable() and compares rows, so a
tree that only LINQ to Objects can run passes. These scenarios also check
the tree: the query must stay on its provider, and must not contain what a
C# expression lambda never does - a block, a parameter nothing binds, a
call into FSharp.Core, a join nested in a collection selector, a member
read from a tuple built without naming its members.

A scenario names what is wrong with its tree today. The name has to be
removed by the change that makes it untrue, because a scenario fails when
its tree is cleaner than it says.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Selecting a struct tuple failed with "unreachable": the tuple was replaced
by an anonymous object, but only reference tuples had an anonymous object
type to be replaced by, and the way back always built a reference tuple.
Both kinds now share the anonymous object of their arity, and the way back
builds the kind the query asked for.

Taking a struct tuple apart in a lambda failed with a tuple index out of
range, because a struct tuple has fields where a reference tuple has
properties.

Fixes dotnet#15522

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The shape tests followed only the first source of each operator. Each lambda
parameter is now tied to the sequence it stands for, so the check also covers
the inner input of a join, the second parameter of a join result selector,
a collection selector of SelectMany, an identity Select, a chain inside the
lambda of another operator and a query captured from earlier.

A value scenario now has to end in a Queryable operator that gives the value:
a count made over the rows of the provider leaves it. The validator itself is
tested on trees built by hand.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A value scenario was accepted when the last execution at the provider was an
operator that gives a value. A count made by the client over rows taken from
the provider passed whenever a function it ran asked the provider for a count
of its own, because that count came last. Now every execution has to be such
an operator, and so does the enumeration of a source taken as it is, which is
recorded as an execution too.

The violations of a tree are compared with every occurrence: a tree with one
block was equal to a tree with two. The scenarios that have several of a kind
are pinned with their number.

The validator tests build a tree with an invoked lambda and one with an
unbound parameter, which no scenario has, and the two client counts.

The release note names the error a struct tuple gave on main.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…y what they hold

The shape tests did not see an unnamed element that passes through a result
selector giving one of its parameters back, in SelectMany, Join and Zip, and
they skipped the element parameter of Aggregate together with the accumulated
value. Both are checked now. A function with a seed takes the accumulated value
and an element, and one without a seed takes two elements.

A scenario compared an entity by the text it makes of itself, which names its
key only: a result with the right key and the wrong name or age passed. An
object of a class of the scenarios is now compared by every property that gives
a plain value.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
String.Contains(string, StringComparison) does not exist on .NET Framework, so
the test project failed to compile for net472 and all the Windows legs of the CI
went red. IndexOf with an ordinal comparison does the same on both.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reflection promises no order of the properties of a class. On .NET Framework one
scenario listed the properties of the first order it rendered in another order
than those of the three after it, so the expected and the actual rows of the
same orders differed as text. The names decide the order now.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
join, groupJoin and leftOuterJoin repeated the same translation with
two differences: the LINQ operator they call and the conversion the
inner result goes through. They are now recognized by one active
pattern and translated by one function that takes the outer source
already translated, and the translation of a for body takes its
source, conversion and variable as arguments. This is a refactoring
without a change in behavior; the next commit reuses both pieces to
translate a for or a join that follows a for into a flat chain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A query over an IQueryable that went on after two fors, with a third
for, a yield!, or a join, groupJoin or leftOuterJoin, nested the rest
of the query inside the collection selector of the first SelectMany.
EF Core translates that only as a lateral join (APPLY), which SQLite
and several other databases lack, and cannot translate a join nested
there at all, so two left joins in a row failed or ran on the client.

The translation now pairs the elements of the first two sources, as
C# does with its transparent identifiers, and continues the query from
the pairs: a further for becomes the next SelectMany and a join the
next Join or GroupJoin of the main chain. A tuple's elements go into
the pair one by one, as they do after two fors. The rewrite applies
only when the query is over an IQueryable and a join's inner source
and key do not use the variable of the outer for; everything else is
translated as before.

What follows a leftOuterJoin is also translated as before. Its
elements hold the group already put through DefaultIfEmpty, a sequence
EF Core cannot carry along the main chain, while it does translate the
nested form with a lateral join on SQL Server and PostgreSQL.

Fixes dotnet#8459

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
leftOuterJoin applied DefaultIfEmpty to the group inside the GroupJoin
result, so the element the join produced carried an already-defaulted
sequence. C# carries the group as GroupJoin produced it and applies
DefaultIfEmpty in the SelectMany that reads it, and that is the shape
EF Core, linq2db and other providers recognize as a left join. With
the F# shape EF Core managed one leftOuterJoin, but not a second one:
the defaulted group of the first had to travel through the second
GroupJoin, and it cannot.

The group now stays as the join produced it wherever the query only
passes it on from one tuple to the next, and DefaultIfEmpty is applied
where the query reads it. The elements are the same, and two
leftOuterJoins in a row become two SQL LEFT JOINs. What changes is the
identity of the group: each read makes its own DefaultIfEmpty sequence
instead of sharing the one made per joined row, which a query that
compares the group itself by reference can observe over LINQ to
Objects.

Fixes dotnet#6552

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xperiandri
xperiandri force-pushed the fix/query-left-outer-join branch from 71691e9 to bef1e23 Compare November 8, 2026 02:24

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New

Development

Successfully merging this pull request may close these issues.

1 participant